The present invention relates generally to audio signal processing and waveform processing, and the modification of harmonic content of periodic audio signals and more specifically to methods for dynamically altering the harmonic content of such signals for the purpose of changing their sound or perception of their sound.
Many terms used in this patent are collected and defined in this section.
The quality or timbre of the tone is the characteristic which allows it to be distinguished from other tones of the same frequency and loudness or amplitude. In less technical terms, this aspect gives a musical instrument its recognizable personality or character, which is due in large part to its harmonic content over time.
Most sound sources, including musical instruments, produce complex waveforms that are mixtures of sine waves of various amplitudes and frequencies. The individual sine waves contributing to a complex tone, when measured in finite disjointed time periods, are called its partial tones, or simply partials. A partial or partial frequency is defined as a definitive energetic frequency band, and harmonics or harmonic frequencies are defined as partials which are generated in accordance with a phenomenon based on an integer relationship such as the division of a mechanical object, e.g., a string, or of an air column, by an integral number of nodes. The tone quality or timbre of a given complex tone is determined by the quantity, frequency, and amplitude of its disjoint partials, particularly their amplitude proportions relative to each other and relative frequency to others (i.e., the manner in which those elements combine or blend). Frequency alone is not a determining factor, as a note played on an instrument has a similar timbre to another note played on the same instrument. In embodied systems handling sounds, partials actually represent energy in a small frequency band and are governed by sampling rates and uncertainty issues associated with sampling systems.
Audio signals, especially those relating to musical instruments or human voices, have characteristic harmonic contents that define how the signals sound. Each signal consists of a fundamental frequency and higher-ranking harmonic frequencies. The graphic pattern for each of these combined cycles is the waveform. The detailed waveform of a complex wave depends in part on the relative amplitudes of its harmonics. Changing the amplitude, frequency, or phase relationships among harmonics changes the ear's perception of the tone's musical quality or character.
The fundamental frequency (also called the 1st harmonic, or f1) and the higher-ranking harmonics (f2 through fN) are typically mathematically related. In sounds produced by typical musical instruments, higher-ranking harmonics are mostly, but not exclusively, integer multiples of the fundamental: The 2nd harmonic is 2 times the frequency of the fundamental, the 3rd harmonic is 3 times the frequency of the fundamental, and so on. These multiples are ranking numbers or ranks. In general, the usage of the term harmonic in this patent represents all harmonics, including the fundamental.
Each harmonic has amplitude, frequency, and phase relationships to the fundamental frequency; these relationships can be manipulated to alter the perceived sound. A periodic complex tone may be broken down into its constituent elements (fundamental and higher harmonics). The graphic representation of this analysis is called a spectrum. A given note's characteristic timbre may be represented graphically, then, in a spectral profile.
While typical musical instruments often produce notes predominantly containing integer-multiple or near integer-multiple harmonics, a variety of other instruments and sources produce sounds with more complex relationships among fundamentals and higher harmonics. Many instruments create partials that are non-integer in their relationship. These tones are called inharmonicities.
The modern equal-tempered scale (or Western musical scale) is a method by which a musical scale is adjusted to consist of 12 equally spaced semitone intervals per octave. The frequency of any given half-step is the frequency of its predecessor multiplied by the 12th root of 2 or 1.0594631. This generates a scale where the frequencies of all octave intervals are in the ratio 1:2. These octaves are the only consonant intervals; all other intervals are dissonant.
The scale's inherent compromises allow a piano, for example, to play in all keys. To the human ear, however, instruments such as the piano accurately tuned to the tempered scale sound quite flat in the upper register because harmonics in most mechanical instruments are not exact multiples and the “ear knows this”, so the tuning of some instruments is “stretched,” meaning the tuning contains deviations from pitches mandated by simple mathematical formulas. These deviations may be either slightly sharp or slightly flat to the notes mandated by simple mathematical formulas. In stretched tunings, mathematical relationships between notes and harmonics still exist, but they are more complex. The relationships between and among the harmonic frequencies generated by many classes of oscillating/vibrating devices, including musical instruments, can be modeled by a functionfn=f1×G(n)where fn is the frequency of the nth harmonic, and n is a positive integer which represents the harmonic ranking number. Examples of such functions are                a) fn=f1×n        b) fn=f1×n×[1+(n2−1)β]1/2 where β is constant which depends on the instrument or on the string of multiple-stringed devices, and sometimes on the frequency register of the note being played.        
An audio or musical tone's perceived pitch is typically (but not always) the fundamental or lowest frequency in the periodic signal. As previously mentioned, a musical note contains harmonics at various amplitude, frequencies, and phase relationships to each other. When superimposed, these harmonics create a complex time-domain signal. The differing amplitudes of the harmonics of the signal give the strongest indication of its timbre, or musical personality.
Another aspect of an instrument's perceived musical tone or character involves resonance bands, which are certain fragments or portions of the audible spectrum that are emphasized or accented by an instrument's design, dimensions, materials, construction details, features, and methods of operation. These resonance bands are perceived to be louder relative to other fragments of the audible spectrum.
Such resonance bands are fixed in frequency and remain constant as different notes are played on the instrument. These resonance bands do not shift with respect to different notes played on the instrument. They are determined by the physics of the instrument, not by the particular note played at any given time.
A key difference between harmonic content and resonance bands lies in their differing relationships to fundamental frequencies. Harmonics shift along with changes in the fundamental frequency (i.e., they move in frequency, directly linked to the played fundamental) and thus are always relative to the fundamental. As fundamentals shift to new fundamentals, their harmonics shift along with them.
In contrast, an instrument's resonance bands are fixed in frequency and do not move linearly as a function of shifting fundamentals.
Aside from a note's own harmonic structure and the instrument's own resonance bands, other factors contributing to an instrument's perceived tone or musical character entail the manner in which harmonic content varies over the duration of a musical note. The duration or “life span” of a musical note is marked by its attack (the characteristic manner in which the note is initially struck or sounded); sustain (the continuing characteristics of the note as it is sounded over time); and decay (the characteristic manner in which the note terminates—e.g., an abrupt cut-off vs. a gradual fade), in that order.
A note's harmonic content during all three phases—attack, sustain, and decay—give important perceptual keys to the human ear regarding the note's subjective tonal quality. Each harmonic in a complex time-domain signal, including the fundamental, has its own distinct attack and decay characteristics, which help define the note's timbre in time.
Because the relative amplitude levels of the harmonics may change during the life span of the note in relation to the amplitude of the fundamental (some being emphasized, some de-emphasized), the timbre of a specific note may accordingly change across its duration. In instruments that are plucked or struck (such as pianos and guitars), higher-order harmonics decay at a faster rate than lower-order harmonics. By contrast, on instruments that are continually exercised, including wind instruments (such as the flute) and bowed instruments (such as the violin), harmonics are continually generated.
On a guitar, for example, the two most influential factors, which shape the perceived timbre, are: (1) the core harmonics created by the strings; and (2) the resonance band characteristics of the guitar's body.
Once the strings have generated the fundamental frequency and its associated core set of harmonics, the body, bridge, and other components come into play to further shape the timbre primarily by its resonance characteristics, which are non-linear and frequency dependent. A guitar has resonant bands or regions, within which some harmonics of a tone are emphasized regardless of the frequency of the fundamental.
A guitarist may play the exact same note (same frequency, or pitch) in as many as six places on the neck using different combinations of string and fret positions. However, each of the six versions will sound quite distinct due to different relationships between the fundamental and its harmonics. These differences in turn are caused by variations in string composition and design, string diameter and/or string length. Here, “length” refers not necessarily to total string length but only to the vibrating portion which creates musical pitch, i.e., the distance from the fretted position to the bridge. The resonance characteristics of the body itself do not change, and yet because of these variations in string diameter and/or length, the different versions of the same pitch sound noticeably different.
In many cases it is desired to affect the timbre of an instrument. Modern and traditional methods do so in a rudimentary form with a kind of filter called a fixed-band electronic equalizer. Fixed-band electronic equalizers affect one or more specified fragments, or bands, within a larger frequency spectrum. The desired emphasis (“boost”) or de-emphasis (“cut”) occurs only within the specified band. Notes or harmonics falling outside the band or bands are not affected.
A given frequency can have any harmonic ranking depending on its relationship relative to the changing fundamental. A resonant band filter or equalizer recognizes a frequency only as being inside or outside its fixed band; it does not recognize or respond to that frequency's harmonic rank. The device cannot distinguish whether the incoming frequency is a fundamental, a 2nd harmonic, a 3rd harmonic, etc. Therefore, the effects of fixed-band equalizers do not change or shift with respect to the frequency's rank. The equalization remains fixed, affecting designated frequencies irrespective of their harmonic relationships to fundamentals. While the equalization affects the levels of the harmonics which does significantly affect the perceived timbre, it does not change the inherent “core” harmonic content of a note, voice, instrument, or other audio signal. Once adjusted, whether the fixed-band equalizer has any effect at all depends solely upon the frequency itself of the incoming note or signal. It does not depend upon whether that frequency is a fundamental (1st harmonic), 2nd harmonic, 3rd harmonic, or some other rank.
Some present day equalizers have the ability to alter their filters dynamically, but the alterations are tied to time cues rather than harmonic ranking information. These equalizers have the ability to adjust their filtering in time by changing the location of the filters as defined by user input commands. One of the methods of the present invention, may be viewed as a 1000-band or more graphic equalizer, but is different in that the amplitude and the corresponding affected frequencies are instantaneously changing in frequency and amplitude and/or moving at very fast speeds with respect to frequency and amplitude to change the harmonic energy content of the notes; and working in unison with a synthesizer adding missing harmonics and all following and anticipating the frequencies associated with the harmonics set for change.
The human voice may be thought of as a musical instrument, with many of the same qualities and characteristics found in other instrument families. Because it operates by air under pressure, it is fundamentally a wind instrument, but in terms of frequency generation the voice resembles a string instrument in that multiple-harmonic vibrations are produced by pieces of tissue whose vibration frequency can be varied by adjusting their tension.
Unlike an acoustic guitar body, with its fixed resonant chamber, some of the voice's resonance bands are instantly adjustable because certain aspects of the resonant cavity may be altered by the speaker, even many times within the duration of a single note. Resonance is affected by the configuration of the nasal cavity and oral cavity, the position of the tongue, and other aspects of what in its entirety is called the vocal tract.